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ARE  EXAM  SCORES  AND  OTHER  ADVANCEMENT 
SYSTEM  COMPONENTS  RELATED  TO  NAVY 
DUTY  STATUS? 

Darlene  Barrow,  Robert  Doucette,  and  Dr.  J.  Michael  Lentz 

This  presentation  will  compare  the  major  Navy  Duty  Status  categories  on  the  basis  of 
their  exam  scores  and  other  advancement  system  components.  The  major  Duty  Status 
groups  include,  Inactive  Duty  Reservists,  active  duty  Canvasser/Recruiters,  active  duty 
Training  and  Administration  of  Reserves,  and  regular  Active  Duty.  Exam  performance 
comparisons  will  be  based  on  the  same  rating  exams.  Other  advancement  system 
components  (award  points  and  performance  marks)  will  also  be  compared.  A  discussion 
of  these  results  and  their  implications  for  different  pay  grades  (E-4/5/6/7)  will  be 
presented. 

The  Naval  Enlisted  Advancement  System  (NEAS)  scores  exams  and  advances  E-4 
candidates  in  three  different  duty  groups  and  E?5  through  E-7  candidates  in  four 
different  duty  groups,  to  the  next  higher  paygrade.  These  four  groups  have  different  type 
assignments: 

a.  Inactive  duty  Reservists  (IR)  personnel  -  sailors  who  work  in  their  rate  that  have 
training  time  and  facility  limitations,  and  are  assigned  to  a  training  site  relative  to 
the  sailor’s  location 

b.  Active  duty  Canvasser/Recruiters  (CVR)  personnel  -  sailors  on  temporary 
assignment  that  recruit  people  for  enlistment  instead  of  working  in  their  rate 

c.  Active  duty  Training  and  Administration  of  Reserves  (TAR)  personnel  -  sailors 
on  continuous  assignment  that  provide  training  support  to  naval  reservists 

d.  Regular  Active  duty  (USN/R)  personnel  -  sailors  on  continuous  assignment  who 
work  in  their  rate 

Active  duty  Naval  Enlisted  Advancement  candidates  for  E-4  groups  (c)  and  (d)  and  E-5 
through  E-7  groups  (b),  (c),  and  (d)  participate  in  the  same  rating  exams  on  scheduled 
paygrade  dates.  IR  candidates  take  the  same  rating  exams  as  active  duty  candidates,  but 
at  a  later  date. 

Figure  1 .  shows  the  raw  score  (RS)  to  standard  score  (SS)  conversion  model  used  by 
NEAS  for  a  150  item  test.  The  IR  candidates  utilize  the  same  exam  rate  raw  score  to 
standard  score  conversion  file  established  by  the  Active  Duty  candidates. 

Raw  score  pass  cut  points  for  a  four-choice  150  item  test  are  derived  using  the  binomial 
distribution,  where  n/4  =  (150)/4  =  38  is  the  mean  and  (npq)m  =  [(150)(0.25)(0.75)]1/2  = 
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5  is  the  standard  deviation.  The  raw  score  cut  point  by  paygrade  is 


E-4 

p  +  ?  =  38  +  5  =  43 

E-5 

p  +  2?  =  38  +  10  =  48 

E-6/7 

p  +  3?  =  38+  15  =  53 

For  our  purpose,  past  rating  exams  for  paygrades  E-4/5/6  (Series  156)  and  for  paygrade 
E?7  (Series  154)  were  used  for 

analysis.  Table  1.  shows  the  relationship  between  the  Active  Duty  and  IR  administration 
dates. 
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TABLE  1.  ADMINISTRATION  DATES 


Exam 

Paygrade 

Series 

Active  Duty 
Exam  Date 

IR  Exam 
Date 

E-4/5/6 

156 

9-97 

2-98 

E-7 

154 

1-97 

2-98 

For  the  duty  groups  by  pay  grade,  we  hypothesize  the  following: 

(a)  H0:  There  are  no  differences  in  exam  Standard  Score  (SS)  means. 

(b)  H0:  There  are  no  differences  in  exam  SS  means  for  advanced  candidates. 

(c)  H0:  There  are  no  differences  in  Performance  Mark  (PMK)  means. 

(d)  H0:  There  are  no  differences  in  Award  point  (AWD)  means. 

METHOD 

One-way  Analysis  of  Variance  tests  were  conducted  to  determine  if  there  were  any 
significant  differences  between  group  means  for  each  null  hypothesis.  SPSS  (Statistical 
Package  for  the  Social  Sciences)  was  used  to  analyze  the  data.  For  each  null  hypothesis 
that  was  rejected  by  the  analysis  of  variance  procedure,  SPSS  Bonferroni  multiple 
comparison  post  hoc  tests  were  applied  between  each  set  of  means. 

RESULTS 

All  four  null  hypotheses  were  rejected  for  all  paygrades.  Significant  differences  between 
duty  group  means  are  shown  in  Tables  2  through  5. 


TABLE  2.  STANDARD  SCORE  STATS  FOR  TOTAL  CANDIDATES 

E-4 

Count 

Mean 

Standard 

Deviation 

IR 

CVR 

TAR 

USN/R 

IR 

586 

43.6 

9.8 
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CVR 

— 

— 

— 

TAR 

522 

53.1 

9.6 

USN/R 

23883 

49.9 

10.0 

Total 

24991 

49.8 

10.0 

E-5 

Count 

Mean 

Standard 

Deviation 

IR 

5153 

40.6 

9.2 

CVR 

12 

41.5 

8.0 

TAR 

1144 

48.3 

9.3 

USN/R 

45538 

50.0 

10.0 

Total 

51847 

49.0 

10.3 

E-6 

Count 

Mean 

Standard 

Deviation 

IR 

8151 

38.2 

9.6 

CVR 

104 

38.7 

9.0 

TAR 

2380 

45.7 

9.6 

USN/R 

38022 

50.0 

10.0 

Total 

48657 

47.8 

10.9 

IR  CVR  TAR  USN/R 


E-7 

Count 

Mean 

Standard 

Deviation 

IR 

6959 

38.4 

9.9 

CVR 

105 

33.0 

8.7 

TAR 

2684 

45.6 

10.1 

USN/R 

38580 

50.0 

9.9 

Total 

48328 

48.0 

10.7 

NOTE:  *  (p  <  0.05)  and  **  (p  <  0.01)  indicates  significant  differences  which  are  shown 
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in  the  lower  triangle. 


TABLE  3.  STANDARD  SCORE  STATS  FOR  ADVANCED  CANDIDATES 


IR 


CVR 


TAR 


USN/R 


Total 


Count  Mean 


Standard 

Deviation 


IR  CVR  TAR  USN/R 


E-5 

Count 

Mean 

Standard 

Deviation 

IR 

737 

47.6 

9.1 

CVR 

9 

43.8 

7.9 

TAR 

332 

54.5 

8.4 

USN/R 

6594 

60.7 

7.9 

Total 

7672 

59.2 

9.0 

E-6 

Count 

Mean 

Standard 

Deviation 

IR 

458 

48.4 

9.1 

CVR 

4 

52.5 

9.7 

TAR 

223 

57.8 

8.4 

USN/R 

2894 

63.5 

7.0 

Total 

3579 

61.2 

9.0 

E-7 

Count 

Mean 

Standard 

Deviation 

IR 

603 

44.6 

8.8 
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CVR 

2 

48.0 

8.5 

TAR 

184 

55.0 

8.3 

USN/R 

2805 

57.7 

7.3 

Total 

3594 

55.4 

9.0 

NOTE:  *  (p  <  0.05)  and  **  (p  <  0.01)  indicates  significant  differences  which  are  shown 
in  the  lower  triangle 


TABLE  4.  PERFORMANCE  MARK  STATS  FOR  TOTAL  CANDIDATES 


IR 


CVR 


TAR 


USN/R 


Total 


Count  Mean 


584  3.645 


522  I  3.710 


Standard 

Deviation 


IR  CVR  TAR  USN/R 


23881 


3.712 


24987  3.710 


0.139 


0.158 


0.156  ** 


0.156 


E-5 

Count 

Mean 

Standard 

Deviation 

IR 

5144 

3.688 

0.139 

CVR 

12 

3.642 

0.100 

TAR 

1144 

3.733 

0.157 

USN/R 

45536 

3.731 

0.146 

Total 

51836 

3.727 

0.146 

E-6 

Count 

Mean 

Standard 

Deviation 

IR 

8140 

3.779 

0.132 

CVR 

104 

3.745 

0.139 

TAR 

2378 

3.839 

0.114 
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USN/R 

38017 

3.844 

** 

** 

Total 

0.118 

E-7 

Count 

Mean 

Standard 

Deviation 

IR 

CVR 

TAR 

USN/R 

IR 

6948 

3.833 

0.120 

CVR 

105 

3.871 

0.067 

TAR 

2682 

3.930 

0.064 

USN/R 

38576 

3.929 

0.070 

Total 

48311 

3.915 

0.086 

NOTE:  (p  <  0.05)  and  **  (p  <  0.01)  indicates  significant  differences  which  are  shown  in 
the  lower  triangle. 
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TABLE  5.  AWARD  STATS  FOR  TOTAL  CANDIDATES 


Mean 


0.9 


Standard 

Deviation 


E-5 

Count 

IR 

5153 

CVR 

12 

TAR 

1144 

USN/R 

45538 

Total 

51847 

Standard 

Deviation 


E-6 

Count 

Mean 

Standard 

Deviation 

IR 

8151 

5.3 

2.6 

CVR 

104 

7.3 

3.2 

TAR 

2380 

6.7 

2.7 

USN/R 

38022 

7.5 

2.7 

Total 

48657 

7.1 

2.8 

IR  CVR 


NOTE:  **  (p  <  0.01)  indicates  significant  differences  which  are  shown  in  the  lower 
triangle. 
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DISCUSSION 

Of  considerable  interest  and  concern  is  percent  of  candidates  failing  their  enlisted 
advancement  exam.  (Refer  to  Table  6.)  The  USN/R  and  TAR  groups  have  relatively 
low  fail  ratios,  5.14%  or  less.  The  CVR  fail  percents  of  25  to  39  are  alarming.  The  very 
significant  loss  of  professional  rating  knowledge  associated  with  a  brief  2  to  3  year 
recruiting  tour  needs  further  investigation.  Of  the  IR  duty  group,  the  E-4  exam 
candidates  have  a  relatively  low  fail  ratio,  4.61%.  However,  the  IR  fail  percent 
dramatically  increases  as  pay  grade  increases.  By  comparison,  the  USN/R  and  TAR 
groups  have  only  a  modest  increase  in  fail  ratios  with  increasing  paygrade.  The  mean 
exam  standard  scores  are  in  general  congruence  with  the  ‘fail’  data. 


TABLE  6.  PERCENT  FAILS  BY  DUTY  GROUP 

IR 

CVR 

TAR 

USN/R 

E-4 

4.61 

— 

0.19 

1.97 

E-5 

16.77 

25.00 

2.27 

3.01 

E-6 

28.99 

25.96 

5.13 

3.43 

E-7 

22.00 

39.05 

5.14 

3.36 

CONCLUSIONS 

It  is  possible  that  with  the  progressive  passage  of  time  plus  very  limited  exposure  to  their 
navy  rating,  IR  and  CVR  sailors  lose  much  of  their  functional  knowledge.  The  data 
seem  to  suggest  that  reservists  and  canvasser/recruiters  need  a  more  active  continuing 
education  program. 

It  should  be  pointed  out  that  sample  size  by  paygrade  for  the  CVR  group  is  small.  The 
average  exam  performance  of  the  CVR  and  IR  duty  groups  are  low  and  are  of  concern. 
From  the  perspective  of  test  design,  the  standard  score  results  are  encouraging.  The 
exams  are  not  intended  to  be  ‘book  tests.’  Instead,  our  goal  has  always  been  to  test 
actual  on-the-job  working  knowledge  and  experience.  The  results  suggest  that  sailors 
working  everyday  in  their  profession  (rate)  are  on  average  performing  much  better  than 
those  that  do  not. 

Although  there  are  some  statistically  significant  differences  in  performance  marks 
(evaluations)  between  groups,  there  is  little  functional  significance  to  these  differences. 

In  a  similar  vein,  the  award  point  comparisons  show  significant  differences  in  some 
means  between  groups  at  pay  grades  E-4  and  E-5.  However,  these  differences  coupled 
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,  * 


with  the  low  variability  in  each  group  are  not  considered  to  be  substantial  for  the  award 
component.  But,  at  paygrade  E-6,  the  IR  award  point  mean  is  significantly  less  than 
each  of  the  other  duty  groups.  The  magnitude  of  these  differences  together  with  each 
group’s  variability  are  considered  to  be  substantial  for  the  award  component. 

The  results  of  this  study  suggest  that  all  three  measures,  exam  standard  score, 
performance  mark,  and  award  points,  are  related  to  navy  duty  group  status.  On  the 
average,  sailors  (USN/R  and  TAR)  with  continuous  training  and  facility  access  perform 
better  than  sailors  (IR  and  CVR)  with  limited  facility  access  and  training  time  (full-time 
civilian  job  or  recruiting  full-time). 
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